Looks good to me, only thing I was concerned is backward compatibility. I know it is difficult to have that with these changes, but still I would like to give some more thoughts to it.

The thing that our main task here was to remove cascading updates that don't work well with our V5 ES/DB architecture. So backward compatibility got broken at the moment when we removed the cascading updates. The fact that we introduced a new bulk milestone updates doesn't harm.

We can try to achieve the backward compatibility here, though I think it would make the code very knotted. Even before the logic was quite tricky, and removing cascading updates is a good step in making logic more straightforward here. But keeping both ways might make it even more cumbersome.

Do we have some reasons for keeping backward compatibility? I know that other teams started using it. Though as timelines with milestones are completely outside of the project object, I hope other teams didn't start to load them. So if it's possible maybe we can try to confirm with them, that they don't use them yet.

Btw, we also have some plans to refactor milestone spans to points as per appirio-tech/connect-app#3299 which could also break backward compatibility.

Thanks for detailed explanation and it is clear now to me and it would be better for future to track back why we did this change and why it broke the backward compatibility.

We can try to achieve the backward compatibility here, though I think it would make the code very knotted.

Though, I am now convinced to do this without backward compatibility, I am still trying to think from future considerations when we would have many other consumers of the projects and timeline apis. Just trying to brainstorm the all possible ways to handle such situations in future. In this particular case, we might have backward compatibility for some time in future (After which it can be deprecated easily):

• Have versions for the timeline just like we have for projects
• For the timelines that are related to the projects, we can upgrade them to the latest version (say 2)
• For timelines other than projects (which is technically zero at present), we can keep them to the current version (say 1)
• Now for the version 1 of the timelines we can move that cascading logic to an event handler i.e. an event handler can listen for milestone update in v1 timeline and can do the cascaded updates for the timeline. And later we can disconnect that event handler when we want to deprecate.

Btw, we also have some plans to refactor milestone spans to points as per appirio-tech/connect-app#3299 which could also break backward compatibility.

Yep, that is why I more worried about two breaking changes. It would have been great if we could have done this in single release. But I think that is difficult to achieve.

Though, I am not convinced to do this without backward compatibility

I understand that not always we can make breaking changes and it would be always a trade-off. If it's harder to update API consumers to new API than keeping backward compatibility then we have to keep backward compatibility no matter how code would become more complex. Though keeping backward compatibility has its cost:

• logic is harder to support
• logic is harder to update: any new logic has to be added in both versions or take them into account if just branch code
• fewer people understand what is going on and need to apply more efforts to get into the context
• cumbersome logic is prone to issues
• this all result in higher expenses

So if it's easier to update all the consumers or there are no consumers, it would better to drop backward compatibility. But if we cannot ask all the consumers to update to the new API we don't have other choice as to support backward compatibility. So it's not impossible, it just has some price to pay.

Now for the version 1 of the timelines we can move that cascading logic to an event handler i.e. an event handler can listen for milestone update in v1 timeline and can do the cascaded updates for the timeline. And later we can disconnect that event handler when we want to deprecate.

This would be the ideal situation which we should try to achieve if we have to keep backward compatibility somewhere.

But in many situations, it's hard to encapsulate all such logic into events.
For example:

• we had automatic milestone update logic, which happened even before we save milestone to the DB. So if we need to keep backward compatibility, we cannot move it to some event as it should be run "synchronously" so we would have to add conditions inside endpoint
• even logic to cascade update timeline cannot be moved to the event completely

  tc-project-service/src/routes/milestones/update.js

  Lines 309 to 321 in 2fd32ce

  	if (needToCascade) {
  	return updateComingMilestones(original, updated)
  	.then(({ originalMilestones, updatedMilestones }) => {
  	// finds the last milestone updated
  	// if no milestone is updated by updateComingMilestones method, it means the current milestone is the last one
  	const lastTimelineMilestone = updatedMilestones.length ? _.last(updatedMilestones) : updated;
  	if (!_.isEqual(lastTimelineMilestone.endDate, timeline.endDate)) {
  	timeline.endDate = lastTimelineMilestone.endDate;
  	timeline.updatedBy = lastTimelineMilestone.updatedBy;
  	return timeline.save().then(() => ({ originalMilestones, updatedMilestones }));
  	}
  	return Promise.resolve({ originalMilestones, updatedMilestones });
  	});

  . The reason for this is because when updating milestone we send an event to the TC Notification service, and we include in this event timeline

  tc-project-service/src/events/busApi.js

  Line 636 in 2fd32ce

  timeline,

  
  If we moved logic to update the timeline from the endpoint to some event, then such an event could happen after sending an event to TC Notification, and as a result, we could get an outdated timeline in the TC Notification.
• These are just 2 big examples, but during working on these milestones refactoring, I came across such places where one logic depends on another one and vise-verse quite many times. So I think if we keep both logics with and without cascading updates, we would end up in many if/else branches across the code and it wouldn't be so easy to support and remove it after.

Just trying to brainstorm the all possible ways to handle such situations in future.

If we don't have a choice other then keeping backward compatibility then having a version for objects as you've suggested is one of the best approaches I think.

Other possible approaches:

• not only have object versions, but also have API versions. As we already use API version for general Topcoder purposes, we can add one more level of API version like:

  • /v5/timelines/v1/{id}/milestones/{id}

  • /v5/timelines/v2/{id}/milestones/{id}
    Or just use the main API version:

  • /v6/timelines/{id}/milestones/{id}

  • /v7/timelines/{id}/milestones/{id}

  • This could make it more obvious for consumers that API is different. This can be combined with versions of timeline/milestone objects.

  • Also, having API versioning could let us introduce separate endpoints with separate code, instead of adding conditions in the existent endpoints. But if later we need to add some features to both versions, we have to implement it 2 times.

• another approach is "duck typing". I. e. don't introduce any version to the object but make code depend on the object values and presence of some properties.
  For example, even though for Projects we have version v3 and v2 we have many more kind of projects than these 2. And it even seems to me, that for Projects we can consider removing version and just rely on the actual data:

  • was created by Product Template (aka v2) or by Project Template (aka v3)
  • has phases or no
  • phases have timelines or no
  • has workstreams or no
  • support uploading files or no

  So as projects are constantly and gradually developing version doesn't represent all the differences (features) of the projects.
  But if something doesn't change so often, having a version might be a good choice.

Yep, that is why I more worried about two breaking changes. It would have been great if we could have done this in single release. But I think that is difficult to achieve.

Yep, there are already many changes across 3 repos. It would hard not to break anything if we did more changes in a single step. In such a way we can control that we don't lose any functionality and it's easier to transit.

.

@vikasrohit let me know if we have to keep backward compatibility for milestones on this step, or this time we can drop it.

Though, I am not convinced to do this without backward compatibility

I meant Though, I am *now* convinced to do this without backward compatibility